class: title-slide, inverse, middle, right background-image: url(https://raw.githubusercontent.com/MiriamLL/Curso_tracking/main/03Figuras/PortadaC.jpg) background-size: cover # Tracking ## **Miriam Lerma**<br> --- ## Intro A typical project. <img src="https://github.com/MiriamLL/R_intro/blob/master/Images/031Program.png?raw=true" height="250" /> Here: - [Import](#read) Import from movebank, csv or excel. - [Tidy](#tidy) Clean your data - [Transform](#transform) Transform the time zone - [Visualise](#visualise) Create a map and include your tracking data --- class:center,middle ## Note <h1>
</h1> This presentation is constantly updated. <img src="https://github.com/MiriamLL/R_intro/blob/master/Images/500UnderConstruction.gif?raw=true" height="100" /> --- name: read class:center,middle <h1>
# Read data from movebank For this part you need to know how to install packages, if you need to learn that first start [here](https://www.miriam-lerma.com/posts/2023-04-18-intro-to-r/). --- ## 1. Install package There is a package called [move](https://cran.r-project.org/web/packages/move/vignettes/browseMovebank.html) that can be use to access data stored in **movebank.** <img src="https://www.movebank.org/cms/img/logo-movebank.png" height="100" /> To install. ```r install.packages('move') ``` To call. ```r library(move) ``` For details go to the [vignette](https://cran.r-project.org/web/packages/move/vignettes/browseMovebank.html). --- ## 1.2. movebank To load data you can enter manually your login information with the argument **movebankLogin**. ```r movebankLogin() ``` It would ask for the login information at the console. <img src="https://github.com/MiriamLL/R_intro/blob/master/Images/401movelogin.png?raw=true" height="200" /> --- ## 1.2. movebank However, it is more convenient to store your login information. <h2>⚠️ </h2> Be careful to not share the script with your login information. <br> <br> For this exercise, I created a username for **demonstration**. The password is the institute, the year and asterisk. ```r my_login<-movebankLogin(username = "Demo", password = "*****") ``` --- ## 1.2. studies For the exercise, we will check studies that have **FTZ** on the name. ```r searchMovebankStudies(x="FTZ", login=my_login) ``` ``` ## [1] "DMM FTZ Great Cormorant Heuwiese" ## [2] "FTZ Barnacle Goose Wadden Sea" ## [3] "FTZ Bar-tailed Godwit" ## [4] "FTZ BfN MONTRACK Black-legged Kittiwake" ## [5] "FTZ BfN MONTRACK Grey Seal" ## [6] "FTZ BfN MONTRACK Northern Gannet" ## [7] "FTZ Black-legged Kittiwake Helgoland" ## [8] "FTZ Brent Goose Wadden Sea" ## [9] "FTZ CICESE Yellow-footed Gull Mexico" ## [10] "FTZ Common Eider" ## [11] "FTZ Common Gull Amrum" ## [12] "FTZ Common Gull Dithmarschen" ## [13] "FTZ Common Gull eastern Baltic Sea" ## [14] "FTZ Common Gull Hamburg" ## [15] "FTZ Common Gull Hamburger Hallig" ## [16] "FTZ Common Gull Kiel" ## [17] "FTZ Common Gull Langenwerder" ## [18] "FTZ Common Gull Oland" ## [19] "FTZ Common Gull Spiekeroog" ## [20] "FTZ Common Murre Helgoland" ## [21] "FTZ Crab Plover Africa" ## [22] "FTZ Curlew" ## [23] "FTZ Curlew East" ## [24] "FTZ CWS Northern Gannet Bonaventure" ## [25] "FTZ: Foraging in lesser black-backed gulls (data from Garthe et al. 2016)" ## [26] "FTZ: Foraging site fidelity and spatial segregation in great black-backed gulls (data from Borrmann et al. 2019)" ## [27] "FTZ Geese Wadden Sea" ## [28] "FTZ Great Black-backed Gull eastern Baltic Sea" ## [29] "FTZ Great Black-backed Gull North Sea" ## [30] "FTZ Great Cormorant Beuchel" ## [31] "FTZ Grey Plover Africa" ## [32] "FTZ Grey Plover Wadden Sea" ## [33] "FTZ Grey Seal Helgoland" ## [34] "FTZ Harbour Seals Wadden Sea" ## [35] "FTZ Herring Gull Amrum" ## [36] "FTZ Herring Gull Beuchel" ## [37] "FTZ Herring Gull Helgoland" ## [38] "FTZ Herring Gull Kiel" ## [39] "FTZ Herring Gull Oland & Langeness" ## [40] "FTZ Herring Gull Spiekeroog" ## [41] "FTZ Herring Gull Trischen" ## [42] "FTZ Herring Gull Walfisch" ## [43] "FTZ Lesser Black-backed Gull Amrum" ## [44] "FTZ Lesser Black-backed Gull Borkum" ## [45] "FTZ Lesser Black-backed Gull Hamburg" ## [46] "FTZ Lesser Black-backed Gull Helgoland" ## [47] "FTZ Lesser Black-backed Gull Juist" ## [48] "FTZ Lesser Black-backed Gull Norderney" ## [49] "FTZ Lesser Black-backed Gull Spiekeroog" ## [50] "FTZ Mediterranean Gull Hamburg" ## [51] "FTZ: Migrating curlews (data from Schwemmer et al. 2021)" ## [52] "FTZ Northern Gannet Helgoland" ## [53] "FTZ Northern Gannet migration Bass Rock" ## [54] "FTZ Northern Gannet migration Skrudur" ## [55] "FTZ Oystercatcher Wadden Sea" ## [56] "FTZ Pied Avocet Wadden Sea" ## [57] "FTZ Shelduck" ## [58] "FTZ Terrestrial Mammals Wadden Sea" ## [59] "FTZ UCN Kelp Gull Chile" ## [60] "FTZ Whimbrel Africa" ## [61] "FTZ Whimbrel Wadden Sea" ## [62] "SJSU FTZ Western Gull SEFI" ``` --- ## 1.2. studies Select one study, and give your study name as it appears in the list. For the example, lets use: ```r my_study<-"FTZ: Foraging in lesser black-backed gulls (data from Garthe et al. 2016)" ``` Check if you have access. ```r getMovebankReferenceTable(study=my_study,login=my_login) ``` If it shows: ```r Error in getMovebank(entity_type = "tag", login, study_id = study) : It looks like you are not allowed to download this data set, have you agreed to the license terms in the web interface? ``` You need to request for permit on the webpage. --- ## 1.3. Permits For getting the permits, go to the page [movebank](https://www.movebank.org/cms/movebank-main). Go to Login/Register. Add your username and password. <img src="https://github.com/MiriamLL/R_intro/blob/master/Images/400movepage.png?raw=true" height="300" /> --- ## 1.3. Permits .pull-left[ Go to **Studies**> Search> Write the name of the study <img src="https://github.com/MiriamLL/R_intro/blob/master/Images/402studies.png?raw=true" height="350" /> ] .pull-right[ Click on the i inside a square> Select Open in studies page <img src="https://github.com/MiriamLL/R_intro/blob/master/Images/403Studies.png?raw=true" height="350" /> ] --- ## 1.3. Permits .pull-left[ Click on the Download> Select Download Data <img src="https://github.com/MiriamLL/R_intro/blob/master/Images/404DownloadData.png?raw=true" height="350" /> ] .pull-right[ Click on I agree for the terms and condition <img src="https://github.com/MiriamLL/R_intro/blob/master/Images/405Terms.png?raw=true" height="250" /> This only needs to be done once. - You don't need to do the permits again. - Now you should be able to download the data directly in R. ] --- ## 1.3. Permits You can download the data here, but then you will need to open it every time. <img src="https://github.com/MiriamLL/R_intro/blob/master/Images/406Save.png?raw=true" height="350" /> --- ## 1.4. Inspect study Back to R, check what data the study contains. ```r getMovebankReferenceTable(study=my_study,login=my_login) ``` If it look alright, you can save information from the study as object. ```r my_study_ref<-getMovebankReferenceTable(study=my_study,login=my_login) ``` Now the information is a data frame in your **Environment**. If we open the data frame we can see that in this study there are 8 individuals. We can use this information to download the tracking data. --- ## 1.5. Download data To download all the data use **getMovebankLocationData**. ```r all_tracks<-getMovebankLocationData(study=my_study, login=my_login) ``` However, this might be too much data, and too much time consuming to download all. <h2> ⏰ </h2> Therefore, we can select specific individuals or specific periods of time. --- ## 1.5. Download data To download one individual, use the information from the data frame to identify which **individual_local_identifier** are on the data. Use the name as an object. ```r my_individual<-"LBBG-Spiekeroog-2010-01" ``` Add the name as an argument in the function. ```r my_individual_track<-getMovebankLocationData(study=my_study, login=my_login, individual_local_identifier=my_individual) ``` The warning is related to having GPS and accelometer data. <h2>🤖 </h2> You should have **my_individual_track** in the environment now. --- ## 1.6. Specific times To select specific times, lets first inspect what information is contained in the data frame using functions from the package **tidyverse**. ```r library(tidyverse) ``` Check location starts ```r first(my_individual_track$timestamp) ``` ``` ## [1] "2010-05-16 18:41:20 UTC" ``` Check location ends ```r last(my_individual_track$timestamp) ``` ``` ## [1] "2010-05-27 13:41:12 UTC" ``` Here we can see that the period of sampling is about one month. --- ## 1.7. Specific times Movebank uses a very specific format for timestamps. So if you want to have data from "2010-05-18 00:00:00 UTC" to "2010-05-20 00:00:00 UTC", you need to provide it like this: ```r my_start<-"201005180000000" my_end<-"201005200000000" ``` To download data from this period, you can add the argument in **getMovebankLocationData**. ```r my_individual_subsampled<-getMovebankLocationData(study=my_study, login=my_login, individual_local_identifier=my_individual, timestamp_start=my_start, timestamp_end=my_end) ``` --- name: read class:center,middle <h1>
# Read data from files For this part you need to know how to set your working directory, if you need to learn that first start [here](https://www.miriam-lerma.com/posts/2023-04-18-intro-to-r/). --- ## 2.1. From csv Many devices will collect data in csv format. To load them into R, the manual way will be clicking on the file and then in **Import Dataset.** <img src="https://raw.githubusercontent.com/MiriamLL/Curso_CIAD/main/Figuras/ImportarBoton.jpg" height="350" /> --- ## 2.2. Several files Most likely you will have several individuals tagged, to load several files, define the folder where the files are. ```r my_directory<-'C://Users//...' ``` Using the package here, simplifies the process. ```r my_directory<-here("Data") ``` To check that you are in the correct folder, ask for a list of the files. To load only **csv** specify the extension. ```r list_files <- list.files(my_directory,pattern="*.csv",full.names=TRUE) list_files ``` To download test data click [
here](https://github.com/MiriamLL/Curso_tracking/tree/main/01Datos). ```r The data comes from: Lerma M, Dehnhard N, Luna-Jorquera G, Voigt CC, Garthe S (2020) Breeding stage, not sex, affects foraging characteristics in masked boobies at Rapa Nui. Behavioral ecology and sociobiology 74: 149. [OpenAccess](https://link.springer.com/article/10.1007/s00265-020-02921-1). ``` --- ## 2.3. Using lapply The function **lapply** allows you to import all the files. ```r my_files <- lapply(list_files,read_csv) ``` If you are lacking of ID inside the files, before joining the files, add an ID. Three options to add ID to your data frames. **1. Manually** Add the information in a column. ```r GPS01<-my_files[1] GPS01$ID<-'GPS01' ``` Recommended only for small lists. ```r GPS02<-my_files[2] GPS02$ID<-'GPS02' ``` --- name: ids ## 2.4. Add ID .pull-left[ **Using the file name and a loop** ```r file_names<-list.files(my_directory,pattern="*.csv",full.names=FALSE) IDs<-gsub('.csv','',file_names) ``` Create a loop function to add a column ID per element of the list. ```r for( i in seq_along(my_files)){ my_files[[i]]$IDs <- IDs[i] } ``` ] .pull-right[ **Using a vector** ```r IDs<-c("ID01","ID02","ID03","ID04","ID05","ID06","ID07","ID08","ID09","ID10") ``` Add a column called IDs to each of the elements of the list. ```r for( i in seq_along(my_files)){ my_files[[i]]$IDs <- IDs[i] } ``` ] --- ## 2.5. Bind When importing the data, you likely have a list of files > to join them in a data frame you can use the function **rbind**. ```r my_tracks <- do.call("rbind",my_files) ``` **Note** that is important to have the same number of column in all the elements of the list. ```r head(my_tracks) ``` --- ## 2.5. Bind The package **purrr** allows you to do the same, joining multiple files. <img src="https://purrr.tidyverse.org/logo.png" height="150" /> Call the package. ```r library(purrr) ``` Instead of a loop we use the function **map_df** from the package purrr ```r my_files<-list.files(".", pattern="*.csv", full.names=TRUE) my_tracks<-map_df(my_files, read.csv) ``` --- ## 2.6. From xlsx If you were using your data in excel, you can use the package readxl to load your data. <img src="https://readxl.tidyverse.org/logo.png" height="150" /> To call the package. ```r library("readxl") ``` Provide your directory and the complete name of the file you want to import. ```r GPS01_excel<- read_excel(paste0(my_directory,("//GPS01.xlsx"))) ``` --- name: tidy class:center,middle <h1>
# 3. Tidy For this part you need to know how to install packages, if you need to learn that first start [here](https://www.miriam-lerma.com/posts/2023-04-18-intro-to-r/). --- ## 3.1. Cleaning Most of the times your data will need some **cleaning**. You can expect to repeat the cleaning several times. For training we can use tracking data from the package **sula** ```r remotes::install_github("MiriamLL/sula") ``` ```r library(sula) GPS_raw<-GPS_raw ``` --- ## 3.1. Cleaning You might be interested in particular columns and for that the function **select** from the package **tidyverse** is very convenient. ```r library(tidyverse) ``` ```r my_tracks<-GPS_raw %>% select(IDs, DateGMT, TimeGMT, Longitude, Latitude) ``` See more in my [page](https://miriamll.github.io/R_intro/DataWrangling_0805.html#68). --- ## 3.2. Format time and date To join columns if date and time are separated you can use the function **paste**. ```r my_tracks$timestamp<-paste(my_tracks$DateGMT,my_tracks$TimeGMT) ``` To tell R this is date time use the function **as.POSIXct**. ```r my_tracks$timestamp_class<-as.POSIXct(strptime(my_tracks$timestamp, "%d/%m/%Y %H:%M:%S"),"GMT") ``` --- ## 3.3. Cut periods Select period you want to cut and include it as an object in format **POSIXct**. ```r Time_start<-as.POSIXct(strptime('02/11/2017 18:10:00', "%d/%m/%Y %H:%M:%S"), "GMT") Time_end<- as.POSIXct(strptime('05/11/2017 14:10:00', "%d/%m/%Y %H:%M:%S"), "GMT") ``` You can include a column to check the correct classification using the argument **mutate**. ```r my_tracks<-mutate(my_tracks, period_class = ifelse(my_tracks$timestamp_class >= Time_start & my_tracks$timestamp_class <= Time_end, "Inside","Outside")) ``` The argument **filter** allows to subset the data frame. ```r my_tracks_period<-my_tracks %>% filter(period_class=='Inside') ``` --- ## 3.3. Cut periods To check if the cut work you can compare your data frames. Number of locations **before** ```r table(my_tracks$IDs) ``` ``` ## ## GPS01 GPS02 GPS03 GPS04 GPS05 GPS06 GPS07 GPS08 GPS09 GPS10 ## 1038 1049 1246 1031 962 1231 1004 1015 931 933 ``` Number of locations **after** ```r table(my_tracks_period$IDs) ``` ``` ## ## GPS01 GPS02 GPS03 GPS04 GPS05 GPS06 ## 986 990 988 983 921 970 ``` --- ## 3.4. Export To export your clean csv to work on them later, give your directory and a name to the new file. ```r library(here) ``` The function **write_csv** allows to export the data as csv. ```r write_csv(my_tracks_period, file=paste0(here('Data'),'/01my_tracks_period.csv')) ``` See more in my [page](https://miriamll.github.io/R_intro/DataWrangling_0805.html#948). --- name: transform class:center,middle <h1>
# 4. Transform For this part knowing functions for data wrangling are very useful, if you need to learn that first, start [here](https://miriamll.github.io/R_intro/DataWrangling_0805.html#1). --- ## 4.1. Correct time Most devices record in a specific time (Coordinated Universal Time (UTC)), which means that if you are outside Europe, you might need to correct your data. The first thing will be to identify which time zone you are on. .center[ <img src="https://upload.wikimedia.org/wikipedia/commons/thumb/d/d2/Timezones2008_UTC%2B0.png/1200px-Timezones2008_UTC%2B0.png" height="350" /> ] --- ## 4.1. Correct time Be sure your timestamp is in POSIXct. ```r my_tracks$timestamp_class<-as.POSIXct(strptime(my_tracks$timestamp, "%d/%m/%Y %H:%M:%S"),"GMT") ``` Be sure you have the correct format. Check if you are using **/** or **-**. .pull-left[ - **y** is **year** - **m** is **month** - **d** is **day**] .pull-right[ - **H** is **Hours** - **M** is **Minutes** - **S** is **Seconds** ] --- ## 4.1. Correct time Some options to correct the time stamp of your data **1. Manually** Subtract the hours difference. ```r my_tracks<-my_tracks %>% mutate(timestamp_correct = timestamp_class - 5*60*60) ``` **2. By time zone** Using the package lubridate ```r library(lubridate) my_tracks$timestamp_tz<-ymd_hms(my_tracks$timestamp_class, tz = "Europe/Amsterdam") my_tracks$timestamp_correct_tz <- with_tz(my_tracks$timestamp_tz,tzone = "America/La_Paz") ``` See more in my [page](https://www.miriam-lerma.com/posts/2022-06-30-time-formats/). --- name: visualise class:center,middle <h1>
# 5. Visualise For this part knowing ggplot2 is very useful, if you need to learn that first, start [here](https://www.miriam-lerma.com/posts/2023-06-06-plotting/). --- ## 5. Plotting To plot the data we can use the package **ggplot2** and the argument **geom_path**. ```r ggplot()+ geom_path(data=my_individual_subsampled, aes(x=location.long, y=location.lat)) ``` <!-- --> See more details on how to plote on the tutorial [here](https://www.miriam-lerma.com/posts/2023-06-06-plotting/). --- ## 5.1. Base map To provide reference of the area, the package **GermanNorthSea** contains shapefiles. Load the package. ```r library(GermanNorthSea) library(sf) ``` Load shapefile into environment. ```r Europe<-st_transform(German_land,4326) ``` <img src="https://www.miriam-lerma.com/posts/2023-06-02-germannorthsea/GNS_hexsticker.png" height="200" /> See all the shapefiles contained in the package [here](https://www.miriam-lerma.com/posts/2023-06-02-germannorthsea/). --- ## 5.1. Base map Create base map. ```r Base_map<-ggplot(data = Europe)+ geom_sf(color = "#e76f51",fill = "#e9c46a")+ coord_sf(xlim = c(3,9), ylim = c(53,56))+ theme_bw()+ theme(panel.background = element_rect(fill = '#a8dadc'),panel.grid.major = element_blank(),panel.grid.minor = element_blank())+ xlab('Longitude')+ylab('Latitude')+ scale_x_continuous(labels = function(x) paste0(x, '\u00B0', "W"))+ scale_y_continuous(labels = function(x) paste0(x, '\u00B0', "N")) Base_map ``` <!-- --> See more details on how to plot in my [page](https://www.miriam-lerma.com/posts/2023-03-04-mapping-in-r/). --- ## 5.2. Add geometry Use the argument **geom_path** to visualize the tracking data. ```r Base_map + geom_path(data=my_individual_subsampled, aes(x=location.long, y=location.lat), color='black') ``` <!-- --> It looks way too small 🔬. --- ## 5.3. Plotly To create interactive plots in R we can use the package plotly. ```r library(plotly) ``` The function **ggplotly** allows to zoom-in. ```r ggplotly(Base_map + geom_path(data=my_individual_subsampled, aes(x=location.long, y=location.lat),color='black')) ```
--- ## 5.4. Static map For the static map select different **coord_sf** to zoom-in. Repeat the process until you are satisfied. ```r Base_map + geom_path(data=my_individual_subsampled, aes(x=location.long, y=location.lat),color='black')+ * coord_sf(xlim = c(7.7,7.9), ylim = c(53.65,53.8)) ``` <!-- --> --- ## Keep learning **Visualizing** - Some inspiration [mapping movement](https://besmovesig.wordpress.com/2021/02/25/mapping-movements-the-art-and-the-science-we-have-the-winners/). **Advance topics** - Presentations in [Animove](https://animove.org/elearning-downloads/) --- name: out # Contact [Index](#index)<br> <br> [Read data](#read)<br> In Spanish [here](https://miriamll.github.io/Curso_tracking/Parte1#1)<br> <br> [Tidy](#tidy)<br> In Spanish [here](https://miriamll.github.io/Curso_tracking/Parte3#1)<br> <br> [Transform](#transform)<br> In Spanish [here](https://miriamll.github.io/Curso_tracking/Parte2#1)<br> <br> [Visualise](#visualise) <br> <br> <br> <br> <br> <br> .right[ This materials are free of use <br> Download the presentation here: [
github](https://github.com/MiriamLL/R_intro) and [
webpage](https://www.miriam-lerma.com/posts/2023-06-06-plotting/) ] .center[ <h3>
[Home ](https://www.miriam-lerma.com/teaching.html) ] <br>